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ABSTRACT 

As the use of fuzzy set theory continues to grow, there, is an increased 
need for methodologies and formalisms to manipulate obtained fuzzy subsets. 
Concepts involving relative position of fuzzy patterns are acknowledged as 
being of high importance in many areas. 

In this paper, we present an approach based on the concept of dominance 
in fuzzy set theory for modelling relative positions among fuzzy subsets of a 
plane. In particular, we define the following spatial relations: to the left 

(right), in front of, behind, above, below, near, far from, and touching. 

This concept has been implemented to define spatial relationships among 
fuzzy subsets of the image plane. Spatial relationships based on fuzzy set 
theory, coupled with a fuzzy segmentation should therefore yield realistic 
results in scene understanding. 


INTRODUCTION 

One of the main difficulties in computer vision is the difference between 
how a human sees a scene and how a computer sees it. A human may see a large 
red building between two trees, but the computer "sees" only a two-dimensional 
array of pixel values. 

To design a user interface for computer vision that can be used without 
extensive special training we have to translate from the computer's view to 
the human's. Ve must segment the image, properly label the objects in it, and 
then describe, the objects both in terms of their absolute properties and in 
terms of their properties relative to each other. 

This paper proposes to examine ways of defining and deriving the relative 
spatial properties of the objects in an arbitrary scene. 

A Need of Fuzzy Set Theory in Computer Vision 

In computer vision, the standard approach to image analysis and 


e Part of this work was done when the author was with Department of Electrical 
ft Computer Engineering, University of Missouri- Columbia, Columbia, MO 65211. 
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recognition is to segment the image into regions and to compute various 
properties of and relationships among these regions. However, the regions are 
not always “crisply" defined. It is sometimes more appropriate to regard them 
as fuzzy subsets of the image. 

In the last several years, there has been increased attention given to 

the use of fuzzy set theory in image segmentation Cl, 2, 3, 41 . 

When the objects in a scene are represented by crisp sets, the 
all-or-nothing definitions of the subsets actually add to the problem of 
generating such relational descriptions. It is our belief that definitions of 

spatial relationships based on fuzzy set theory, coupled with a fuzzy 
segmentation will yield realistic results. 

For the purpose of this work we assume that we deal with an linage of 
objects, that is, the scene has already been segmented and the objects have 
been labelled. The segmentation may be either crisp or fuzzy. 

Using the above considerations the problem may be looked at in three 
different ways: 

i. Given a scene, describe (linguistically) the spatial relations between the 

objects in the scene, 

il. Given a scene and a spatial description of an object, find that object in 
the scene, 

ill. Given the spatial relations between the objects , construct a scene, 

locating the objects so as to satisfy those spatial relations <this is the 
"layout" problem). 

This work concentrates on the first two problems, although the resulting 
definitions of spatial relations will be useful for the "layout" problem. 


SPATIAL RELATIONS AMONG FUZZY SUBSETS 

Spatial relationships between regions in an image play important role 
in scene understanding. Humans are able to quickly ascertain the relationship 
between two objects, for example "B is to the right of A", or "B is in front 
of A", but this has turned out to be a somewhat illusive task for automation 
15, 6, 73. 

When the objects in a scene are represented by crisp sets, the 
all-or-nothing definitions of the subsets actually add to the problem of 
generating such relational descriptions. It is our belief that definitions of 
spatial relationships based on fuzzy set theory, coupled with fuzzy 
segmentation will yield realistic results. 


The Idea of Projections 

This work proposes an initial approach at defining spatial relationships 
among fuzzy subsets of the image plane. 
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The idea is to project the fuzzy subsets onto two orthogonal coordinate 
axes and to utilize fuzzy dominance relations to capture the approximate 
relationships. 

Let A be a fuzzy subset of an image. Then A £ U x V, where U is the first 
spatial coordinate axis and V is the second one. In our case, both U and V are 
subsets of the reals (assumed to be the interval (0, 13 for convenience). Then 
y> is a fuzzy relation in U x V. The projection of A onto U, denoted A^ 

is that fuzzy subset of U given by 
P Au <x> ■ sup < Cx,y> > 

for each x e U. 

A similar equation defines the projection of A onto V, that is 
P Ay Cy> - sup < P A (x,y) > 

for each y € V. 

* 01 
For a fuzzy subset CL of U, the crlevel set C is defined by 

C a m < X € U | > a > 

for a € tO, 1], 

When a ■ 0, the inequality is usually considered to be strict and the C a is 
called the support of C. 


Definitions of Spatial Relations for Fuzzy Objects 

Once the two fuzzy subsets A and 8 are projected onto U and V axes, 
methods must be defined to access their relative position. 

In this paragraph we introduce definitions for spatial relations. 


Definition 1 : Ve say that subset A is to the right of subset B if the 

projection of A onto the U axis dominates the projection of B, while the 
projections onto the V axis are (ideally) identical. In other words p v (a) 

should stay near zero for all a (especially for small a>. 

Similar definitions are suggested for all other spatial relations £13, 141 
The definitions are for antisymmetric and transitive relations, that Is TO 
THE LEFT (RIGHT) OF, IN FRONT OF (BEHIND), ABOVE (BELOV), INSIDE (OUTSIDE). 
They are strict partial order relations (i.e. reflexive, antisymmetric and 
transitive) and every one has a semantic inverse. 

Separation Measure 

Let A y , B y , A v , B y be the projections of A and B onto U and V, 

respectively. Since these projections are fuzzy numbers, their a- level sets 
are intervals, i.e.. 
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.a r .al . or, , 

A ■ CA , A J, etc. 
u u u 

For the projections of A and B onto the U axis, the orseparation of A and B is 
defined by 

S® ■ ( A® - B® )* / < V° ♦ W® >* 

U O O Au Bu 


where 


A® ■ 
u 

C A al ♦ A® 1 } 
u u 

/ 2, 


B a - 
u 

< B® 1 + B ar > 
u u 

\ 

to 

v a - 

Au 

C A"- A al > 
u u 

/ 2, and 

W ■ 

Bu 

<B® r - B®S / 2. 

u u 


Now, S® is the ratio of the square of difference between the midpoints of the 

crlevel sets and the square of the sum of the half-widths of these intervals. 
Similar equations are used for the projection of A and B onto the V axis. 

Definition 2 : We say that A^ and B^ are cr-separated if S® > 1. 

Definition 3 : We say that A^ and B y are a- just separated if S® ■ 1. 

Definition 4 : We say that A y and B y are a-overlapping if S® < 1. 

Theorem 1 : i) A and B are er*separated if and only if A ar < B®\ 

- U u u u 

ii> A and B are a- just separated if and only if A ar ■ B®\ 

u u J u u 

iii> A and B are a-overlappinr if and only if A ar > B al . 
u u u u 

The proof of the theorem can be found in (91. 
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The value of these definitions and theorems is two- fold. First, they 
incorporate the fuzziness in th» description of image regions, i.e., they use 
fuzzy subsets of the plane. Second, they deal with the ambiguity of defining 
spatial relationships in the plane. By this we mean that It is possible that 

parts of the two sets can overlap (small a) and yet be well separated Cor 

large a. 

The values of can get arbitrarily large as the widths of the level set 

Intervals get small. In order to create a fuzzy membership function, we will 

map the Interval 10, ax> into [0, 1] by an “S*shaped function" CIS] as follows. 

For a given a, suppose ■ 10. 0.2] and ■ [0.8, 11. (Recall that we have 

scaled the domain of the image into the unit square). Then S® ■ 16. This 

amount of separation (or more) will be considered complete, i.e., jj(S^) * 1 if 

S® > 16. Also we will require that p(0) ■ 0, jj(l) ■ 0.5 and #/(l 6) ■ 0. Such 
a function is defined in our case by: 


0.5 S 2 


M(S) 


I -0.0022 S 2 + 0.0711 S + 0.4311 

1 1 


0 < S < 1 
1 < S < 16 

S > 16 


The Model for Spatial Relationships 

The model for given spatial relationships can now be defined from the 
fuzzy subsets p and p^ ° f to, 13. For example, to model the relationship “A 

IS TO THE RIGHT OF B", we would like the projection of A onto the U axis to 
dominate that of B; wheares the projections should (ideally) be identical on 

the V axis. That is, p v (a) should stay near zero for all a (especially for 

small a). Similar observations can be made for "ABOVE", and "BELOV". 

Instead of dealing with two fuzzy subsets, p y and p^ can be combined into 

a single set from which the relationship can be determined. Fuzzy set theory 
offers an infinite number of aggregation operators, which, given two pieces of 
evidence (values in 10, 13) can produce essentially any composite value 
between 0 and 1, depending on the type of connective and the parameters 

chosen. Union operators produce values greater than or equal to the maximum of 
the two numbers; Intersection operators give a result less than or equal to 
the minimum; and generalized means fill the gap between the minimum and 
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maximum (111. 

"TO THE RIGHT OF" should therefore be a combination of p ■ and the 

complement of 1 since its large values signify that the level sets of A are 
“above or below" those of B. 

For the experiments described In the next paragraph, we chose a 
generalized mean 


mC Mu , iu v > • IV // ♦ « - V> <1 -p v >V" 


as the aggregation connective 1163. In this way, higher weight can be 
associated with the horizontal component with decreased compensation as the 
level sets diverge vertically. Note, that if P -> 00 , then we have till: 


lim m(M u ,P v >-max(^ u ,p v >. 
p- > 00 


Either the two fuzzy sets and <1 or the single aggregated set 

fj^y can be used to define the relation "A IS TO RIGHT OF B". If a single 

value for the degree to which the two sets satisfy the relation is desired, we 
can construct a fuzzy measure from the sets - such as the integral of the 
fuzzy number, or the output of an ordered weighted average (OVA) (121. An 
alternate approach is to use the curves directly to define a linguistic 
assessment of the relation. Here, it is necessary to define fuzzy sets 
representing terms used in the relation, such as “to the right of", "somewhat 
to the right or*, "barely to the right of", "very to the right of", etc. These 
sets could be defined by the designer of the system, or perhaps, by utilizing 
a group of humans to give relative comparisons of a set of examples. The 
actual curve is then matched., to the closest term available to give the 
linguistic assessment. This process is known as linguistic approximation 1131. 


Results of Sample Systems 

All the definitions and theorems listed above were tested using simulated 
data on a computer workstation. Fuzzy subsets with two-sided drum like shaped 
membership functions on projections were used. The experiments were as 
follows. Let us consider an image containing two fuzzy subsets A and B whose 
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membership functions are identical gaussians, but with different mean 
locations. The set B will be fixed with mean COS, 03). Table 1 shows the 

fuzzy set generated from eight choices of locations for the mean of A 

(assume that the V coordinate for the mean is 05). As can be seen, as the set 
A moves to the right* the fuzzy set fj Increases for all a. Recall that the 

value fj (a) ■ 0.5 represents the Just separated condition. The seven a- values 

are 0.011, 0.135, 0.258, 0.606, 0.796, 0.882, 0.923. They were chosen in o^der 

to get^ the following ranges fron^ the njean of gaussian functions: - 0.4a, - 

03a, - 0.6T45 a, - a, - 1.645 a, - 2a, - 3 a, where a is a standard deviation. 


1 Moan of 

I Projections 

I A 

a i 

a 2 

a 3 

a 4 

a 5 

a 6 

°7 

1 0.525 

.001 

.002 

.003 

.008 

.017 

.031 

.049 

I 0.550 

.014 

.031 

.046 

.125 

.275 

.500 

.516 | 

I 0.575 

.070 

.158 

.234 

.508 

.543 

.579 

.623 J 

| 0.600 

.222 

.500 

.514 

.564 

.622 

.680 

.731 | 

1 0-^25 

.503 

.536 

.558 

.631 

.712 

.788 

.851 

0.650 

.532 

.580 

.609 

.706 

.806 

.891 

.950 

0.675 

.567 

.628 

.665 

.783 

.893 

.968 

.998 

0.700 

.604 

.680 

.724 

.857 

.962 

O 

o 

H 

o 

o 


Table 1. Membership functions generated from the projection of A onto U axis. 


Since the projections onto V for these sets are the same as the 
projections onto U, the fuzzy sets from Table 1 can be used to simulate other 
p lacings of A relative to B, e.g., to the northeast or sout he a s t. Table 2 
shows four cas es for the placement of the center of set A along with the 
aggregated fuzzy set generated from both projections. Generalized mean with V 
m 0.73 and P ■ 2 was used. The first case represents a set A which is east of 
B. Here, the combined values are larger than those for the U projections only. 
In fact, even the smallest a <0.011) gives rise to a membership larger than 
05 (the just separated crossover point). In case 2, the set A has moved to 
the north east of B. The movement north effectively decreases the membership 
in the fuzzy set "A is to the right of B". Cases 3 and 4 depict the situation 
where A is directly above B. As the centers move further apart, the membership 
drops dramatically. 
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a 

a 

a 

a 

a 

mm 



l 

2 

3 

4 

3 

■hi 

Kb 

(. 6 , . 5 ) 








% 

• 222 

,500 

.514 

.564 

.622 

.680 

.731 

1 - 

1.00 

1 .00 

1.00 

1.00 

1.00 

1 .00 

1.00 

"<V V 

0.54 

0.66 

0.67 

0.70 

0.73 

0.77 

0.81 

(. 6 , . 6 ) 









.222 

.500 

.514 

.564 

.622 

.680 

.731 

X - "v 

.778 

.500 

.486 

.436 

.378 

.320 

.269 

m <V V 

0.43 

0.50 

0.51 

0.53 

0.57 

0.61 

0.65 

I (. 5 , . 6 ) 









0.00 

0.00 

0,00 

0.00 

0.00 

0.00 

0.00 1 

1 " "v 

.778 

.500 

.486 

.436 

.378 

.320 

.269 I 

m <V M y ) 

0.39 

0.25 

0.24 

0.22 

0.19 

0.16 

0.13 

I ( . 5 , . 7 ) 








"u 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

I 1 - p 
9 v 

.396 

.320 

.276 

.143 

.038 

0.00 

0.00 

J m< *V 

0.20 

0.16 

0.14 

0.07 

0.02 

0.00 

0.00 


Table 2. Combined membership function for the relation "A is to the right of 
B" <V - 0.75, P - 2). 


If we change either the weight W or the exponent P, we can alter the 
shape of the resultant fuzzy set. For more details see 191. 
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Summary and Conclusions 

A now approach, based on tho concept# of dominance in fuzzy set# theory, 
for modelling spatial relationships among fuzzy subsets of an image has been 
proposed. Simulation results were presented to corroborate the theory and 
demonstrate the power of the approach for image description. 
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